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1 Gestures: SHARKS a large vocabulary shorthand writing system for pen-based 
<g> computers 

Per-Ola Kristensson, Shumin Zhai 

October 2004 Proceedings of the 17th annual ACM symposium on User interface 
software and technology 

Publisher: ACM Press 

Full text available: ^ pdf(321.66 KB) Additional Information: full citation , abstract , references , index terms 

Zhai and Kristensson (2003) presented a method of speed-writing for pen-based 
computing which utilizes gesturing on a stylus keyboard for familiar words and tapping for 
others. In SHARK<sup>2</sup>:, we eliminated the necessity to alternate between the 
two modes of writing, allowing any word in a large vocabulary (e.g. 10,000-20,000 
words) to be entered as a shorthand gesture. This new paradigm supports a gradual and 
seamless transition from visually guided tracing to recall-based ges ... 

Keywords: gesture recognition, shorthand, shorthand recognition, stenography, text 
input 



2 Gesture and interaction: Fluid inking: augmenting the medium of free-form inking with 
gestures 

Robert Zeleznik, Timothy Miller 

June 2006 Proceedings of the 2006 conference on Graphics interface GI '06 
Publisher: Canadian Information Processing Society 

Full text available: ^ pdf(230.04 KB) Additional Information: full citation , abstract , references , index terms 

We present Fluid Inking, a generally applicable approach to augmenting the fluid medium 
of free-form inking with gestural commands. Our approach is characterized by four design 
criteria, including: 1) pen-based hardware impartiality: all interactions can be performed 
with a button-free stylus, the minimal input hardware requirement for inking, and the 
least common denominator device for pen-based systems ranging from PDAs to 
whiteboards; 2) performability: gestures use short sequences of simple ... 

Keywords: button-free, gestures, inking, tablet computing, terminal punctuation 
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June 2006 Proceedings of the 2006 conference on Graphics interface GI '06 
Publisher: Canadian Information Processing Society 

Full text available: ^pdf(350.91 KB) Additional Information: full citation, abstract , references , index terms 

Pen gesture interfaces have difficulty supporting arbitrary multiple-stroke selections 
because lifting the pen introduces ambiguity as to whether the next stroke should add to 
the existing selection, or begin a new one. We explore and evaluate techniques that use a 
non-preferred-hand button or touchpad to phrase together one or more independent pen 
strokes into a unitary multi-stroke gesture. We then illustrate how such phrasing 
techniques can support multiple-stroke selection gestures with tapp ... 

Keywords: gestures, multiple strokes, pen input, phrasing, tablets 

4 Manipulating simulated objects with real-world gestures using a force and position g|$ 
sensitive screen 
Margaret R. Minsky 

January 1984 ACM SIGGRAPH Computer Graphics , Proceedings of the 11th annual 

conference on Computer graphics and interactive techniques SIGGRAPH 
'84, Volume 18 Issue 3 
Publisher: ACM Press 

Full text available: 1|| pdf(1.20 MB) Additional Information: full citation , abstract , references , citings, index 

terms 

A flexible interface to computing environments can be provided by gestural input. We 
describe a prototype system that recognizes some types of single-finger gestures and 
uses these gestures to manipulate displayed objects. An experimental gesture input 
device yields information about single finger gestures in terms of position, pressure, and 
shear forces on a screen. The gestures are classified by a "gesture parser" and used to 
control actions in a fingerpainting program, an int ... 

Keywords: Computers and education, Gesture, Paint programs, Touch-sensitive screen, 
Visual programming 

5 Multi-finger and whole hand gestural interaction techniques for multi-user tabletop g| 
displays 

Mike Wu, Ravin Balakrishnan 

November 2003 Proceedings of the 16th annual ACM symposium on User interface 

software and technology 

Publisher: ACM Press 

Full text available: f B pdffi. 10 MB) A 

mov(3 4 MIN) Additional Information: full citation , abstract , references , citings , index 

wmv(3:4 MIN) 

Recent advances in sensing technology have enabled a new generation of tabletop 
displays that can sense multiple points of input from several users simultaneously. 
However, apart from a few demonstration techniques [17], current user interfaces do not 
take advantage of this increased input bandwidth. We present a variety of multifinger and 
whole hand gestural interaction techniques for these displays that leverage and extend 
the types of actions that people perform when interacting on real physi ... 

Keywords: collaborative and competitive applications, gestures, multi degree-of- freedom 
input, tabletop interaction 
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geometric pattern matching 
Per-Ola Kristensson, Shumin Zhai 

January 2005 Proceedings of the 10th international conference on Intelligent user 

interfaces 
Publisher: ACM Press 

Full text available: ^pdf(219.35 KB) Additional Information: full citation , abstract , references , index terms 

Fitts' law models the inherent speed-accuracy trade-off constraint in stylus typing. Users 
attempting to go beyond the Fitts' law speed ceiling will tend to land the stylus outside 
the targeted key, resulting in erroneous words and increasing users' frustration. We 
propose a geometric pattern matching technique to overcome this problem. Our solution 
can be used either as an enhanced spell checker or as a way to enable users to escape 
the Fitts' law constraint in stylus typing, potentially result ... 

Keywords: Fitts' law, spell checker, stylus keyboard, text input, typing correction, typing 
errors, virtual keyboard 



Input interaction: Shorthand writing on stylus keyboard 
Shumin Zhai, Per-Ola Kristensson 

April 2003 Proceedings of the SIGCHI conference on Human factors in computing 

systems 
Publisher: ACM Press 

Full text available- H!) odf(275 25 KB) Additional Information: full citation , abstract , references , citings , index 
^ terms 

We propose a method for computer-based speed writing, SHARK (shorthand aided rapid 
keyboarding), which augments stylus keyboarding with shorthand gesturing. SHARK 
defines a shorthand symbol for each word according to its movement pattern on an 
optimized stylus keyboard. The key principles for the SHARK design include high efficiency 
stemmed from layout optimization, duality of gesturing and stylus tapping, scale and 
location independent writing, Zipfs law, and skill transfer from tapping to sho ... 

Keywords: handheld devices, mobile, pervasive computing, text input, text-entry 



8 Pointing: Interacting with large displays from a distance with vision-tracked rnulti- 
^ finger gestural input 

^ Shahzad Malik, Abhishek Ranjan, Ravin Balakrishnan 

October 2005 Proceedings of the 18th annual ACM symposium on User interface 

software and technology UIST '05 
Publisher: ACM Press 

Full text available: ^ pdf(1.68 MB) Additional Information: full citation , abstract , references , index terms 

We explore the idea of using vision-based hand tracking over a constrained tabletop 
surface area to perform multi-finger and whole-hand gestural interactions with large 
displays from a distance. We develop bimanual techniques to support a variety of 
asymmetric and symmetric interactions, including fast targeting and navigation to all 
parts of a large display from the comfort of a desk and chair, as well as techniques that 
exploit the ability of the vision-based hand tracking system to provide mu ... 

Keywords: asymmetric, bimanual, from afar, gesture, interaction, large wall, multi-point, 
symmetric, touch surface, two hands, visual touchpad 
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Sachi Mizobuchi, Michiaki Yasumura 

April 2004 Proceedings of the SIGCHI conference on Human factors in computing 

systems 
Publisher: ACM Press 

Full text available: ^ pdf(340.47 KB) Additional Information: full citation , abstract , references , index terms 

Tapping-based selection methods for handheld devices may need to be supplemented 
with other approaches as increasingly complex tasks are carried out using those devices. 
Circling selection methods (such as the Lasso) allow users to select objects on a touch 
screen by circling with a pen. An experimental comparison of the selection time and 
accuracy between a circling method and a traditional tapping style of selection was carried 
out. The experiment used a two dimensional grid (varying in terms ... 

Keywords: gesture input, handheld devices, input and interaction technologies, pen user 
interface, target selection 



10 Short Talks: The sound of one hand: a wrist-mounted bio-acoustic fingertip gesture jj§| 
|k interface 

^ Brian Amento, Will Hill, Loren Terveen 

April 2002 CHI '02 extended abstracts on Human factors in computing systems 
Publisher: ACM Press 

Full text available: ^ pdf(320.04 KB) Additional Information: full citation , abstract , references 

Two hundred and fifty years ago the Japanese Zen master Hakuin asked the question, 
"What is the Sound of the Single Hand?" This koan has long served as an aid to 
meditation but it also describes our new interaction techinique. We discovered that gentle 
fingertip gestures such as tapping, rubbing, and flicking make quiet sounds that travel by 
bone conduction throughout the hand. A small wristband-mounted contact microphone 
can reliably and inexpensively sense these sounds. We harnessed this "soun ... 

Keywords: acoustics, fingers, gestural interfaces, gestures, human-computer interaction, 
mobile devices, wrist 



11 Short Talks: Using a g estural interface toolkit for tactile input to a dynamic virtual 
space 

^ Thecla Schiphorst, Robb Lovell, Norman Jaffe 

April 2002 CHI '02 extended abstracts on Human factors in computing systems 
Publisher: ACM Press 

Full text available: ^]pdf(1 42.51 KB) Additional Information: full citation , abstract , references 

In this paper, we describe the development of a gesture interface toolkit that has been 
applied to an application of tactile gesture recognition within an artificial life environment. 
The goal is to design a gestural semantics of caress, in which qualitative attributes of 
gesture are expressed as a function of tactility. A touch-sensitive tablet capable of 
detecting multiple simultaneous contacts was used to provide a source of tactile gestures 
(stroking, pressing, tapping, wrapping, spreading, ... 

Keywords: CAVE, Laban Effort-Shape analysis, Max/MSP, Tactex MTC, gestural analysis, 
gesture recognition, gesture toolkit, gesture-based interface, immersive environment, 
movement analysis, tactile input, whole hand input 



12 Hands-free multi-scale navigation in virtual environments 

Joseph J. LaViola, Daniel Acevedo Feliz, Daniel F. Keefe, Robert C. Zeleznik 
March 2001 Proceedings of the 2001 symposium on Interactive 3D graphics 



http://portal.acm.org/results.cfm?coll=ACMi&dl=ACM&CFID=6 



12/11/2006 



Results (page 1): tapping and gesturing 



Page 5 of 7 



Publisher: ACM Press 

Full text available: 1|| pdf(1 55.62 KB) Additional Information: full citation , references , citings , index terms 



Keywords: auto rotation, gestural interaction, navigation techniques, virtual reality 



13 Speech, Audio, Gesture: Gestural and audio metaphors as a means of control for 
mobile devices 

Antti Pirhonen, Stephen Brewster, Christopher Holguin 

April 2002 Proceedings of the SIGCHI conference on Human factors in computing 

systems: Changing our world, changing ourselves 
Publisher: ACM Press 

Full text available* pc |f(464 75 KB) Add ' tiona, Information: full citation , abstract , references , citings , index 
^ terms 

This paper discusses the use of gesture and non-speech audio as ways to improve the 
user interface of a mobile music player. Their key advantages mean that users could use 
a player without having to look at its controls when on the move. Two very different 
evaluations of the player took place: one based on a standard usability experiment 
(comparing the new player to a standard design) and the other a video analysis of the 
player in use. Both of these showed significant usability improvements for ... 

Keywords: evaluation, gestures, metaphor, mobile computing, non-speech audio 



14 Small devices 1: AppLens and launchTile: two designs for one-handed thumb use on |j| 
^ small devices 

^ Amy K. Karlson, Benjamin B. Bederson, John SanGiovanni 

April 2005 Proceedings of the SIGCHI conference on Human factors in computing 

systems 
Publisher: ACM Press 

Full text available: ^ pdf(467.29 KB) Additional Information: full citation , abstract , references , index terms 

We present two interfaces to support one-handed thumb use for PDAs and cell phones. 
Both use Scalable User Interface (ScUI) techniques to support multiple devices with 
different resolutions and aspect ratios. The designs use variations of zooming interface 
techniques to provide multiple views of application data: AppLens uses tabular fisheye to 
access nine applications, while LaunchTile uses pure zoom to access thirty-six 
applications. We introduce two sets of thumb gestures, each representing d ... 

Keywords: gestures, mobile devices, notification, one-handed, piccolo, thumb 
navigation, zoomable user interfaces (ZUIs) 



15 Ubiquitous computing: Structuralizinq digital ink for efficient selection j 
Xiang Ao, Junfeng Li, Xugang Wang, Guozhong Dai 

January 2006 Proceedings of the 11th international conference on Intelligent user 

interfaces IUI '06 
Publisher: ACM Press 

Full text available: ^ pdf(393.86 KB) Additional Information: full citation , abstract , references , index terms 

Raw digital ink is informal and unstructured. Its editing, especially its selection, is often 
inefficient. In this paper, we present approaches to structuralize raw digital ink as multiple 
hierarchies to facilitate its selection. First a link model is built to organize ink as a mesh- 
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like structure. Based on the link model, the isolated stroke groups form patches. In each 
patch, textual and graphical areas are separated. Then, each textual area is segmented 
into text lines, and each text line is ... 

Keywords: digital ink, selection gesture, structuralizing, text line extraction, text/graph 
separation 



16 Visual similarity of pen gestures 

A. Chris Long, James A. Landay, Lawrence A. Rowe, Joseph Michiels 
April 2000 Proceedings of the SIGCHI conference on Human factors in computing 

systems 
Publisher: ACM Press 

Full text available: Hi pdf(972 81 KB) Additional Information: full citation , abstract , references , citings, index 

terms 

Pen-based user interfaces are becoming ever more popular. Gestures (i.e., marks made 
with a pen to invoke a command) are a valuable aspect of pen-based Ills, but they also 
have drawbacks. The challenge in designing good gestures is to make them easy for 
people to learn and remember. With the goal of better gesture design, we performed a 
pair of experiments to determine why users find gestures similar. From these 
experiments, we have derived a computational model for predicting perceived gestur ... 

Keywords: multi-dimensional scaling, pen gestures, pen-based user interfaces, 
perception, similarity 



17 Dasher— a data entry interface using continuous gestures and language models 
David J. Ward, Alan F. Blackwell, David J. C. MacKay 

November 2000 Proceedings of the 13th annual ACM symposium on User interface 
software and technology 

Publisher: ACM Press 

Full text available: 1pj pdf(110.32 KB) Additional Information: full citation , references , citings , index terms 



Keywords: adaptive, entry, language, modelling, text 

18 Regular contributions: Human-centered interaction with documents 
Andreas Dengel, Stefan Agne, Bertin Klein, Achim Ebert, Matthias Deller 

v October 2006 Proceedings of the 1st ACM international workshop on Human-centered 
multimedia HCM '06 
Publisher: ACM Press 

Full text available: ^ pdf(767,95 KB) Additional Information: full citation , abstract , references , index terms 

In this paper, we discuss a new user interface, a complementary environment for the 
work with personal document archives, i.e. for document filing and retrieval. We introduce 
our implementation of a spatial medium for document interaction, explorative search and 
active navigation, which exploits and further stimulates the human strengths of visual 
information processing. Our system achieves a high degree of immersion of the user, so 
that he/she forgets the artificiality of his/her environment. T ... 

Keywords: 3D displays, 3D user interface, data glove, gesture recognition, immersion 
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^ disgjays 

^ Maryam Kamvar, Patrick Chiu, Lynn Wilcox, Sandeep Casi, Surapong Lertsithichai 
April 2004 CHI '04 extended abstracts on Human factors in computing systems 
Publisher: ACM Press 

Full text available: pdf(400.76 KB) Additional Information: full citation , abstract , references , index terms 

It is challenging to browse multimedia on mobile devices with small displays. We present 
MiniMedia Surfer, a prototype application for interactively searching a multimedia 
collection for video segments of interest. Transparent layers are used to support browsing 
subtasks: keyword query, exploration of results through keyframes, and playback of 
video. This layered interface smoothly blends the key tasks of the browsing process and 
deals with the small screen size. During exploration, the u ... 

Keywords: multimedia, pen-based computers, small displays, video 



20 Touch-sensing input devices 
Ken Hinckley, Mike Sinclair 

May 1999 Proceedings of the SIGCHI conference on Human factors in computing 
systems: the CHI is the limit 

Publisher: ACM Press 

Full text available' 11| p df(1.23 MB) Additional Information: full citation , abstract , references , citings , index 

' terms 

We can touch things, and our senses tell us when our hands are touching something. But 
most computer input devices cannot detect when the user touches or releases the device 
or some portion of the device. Thus, adding touch sensors to input devices offers many 
possibilities for novel interaction techniques. We demonstrate the TouchTrackball and the 
Scrolling TouchMouse, which use unobtrusive capacitance sensors to detect contact from 
the users hand without requiring pressure or mechan ... 

Keywords: haptic input, input devices, interaction techniques, sensor technologies, 
tactile input, touch-sensing devices 
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